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(57) ABSTRACT 

The diagnosis problem arises when a system’s actual behav- 
ior contradicts the expected behavior, thereby exhibiting 
symptoms (a collection of conflict sets). System diagnosis is 
then the task of identifying faulty components that are 
responsible for anomalous behavior. To solve the diagnosis 
problem, the present invention describes a method for find- 
ing the minimal set of faulty components (minimal diagnosis 
set) that explain the conflict sets. The method includes acts 
of creating a matrix of the collection of conflict sets, and 
then creating nodes from the matrix such that each node is 
a node in a search tree. A determination is made as to 
whether each node is a leaf node or has any children nodes. 
If any given node has children nodes, then the node is split 
until all nodes are leaf nodes. Information gathered from the 
leaf nodes is used to determine the minimal diagnosis set. 
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SYSTEM FOR SOLVING DIAGNOSIS AND 
HITTING SET PROBLEMS 

PRIORITY CLAIM 

This is a non-provisional application claiming the benefit 
of priority of U.S. Provisional Application No. 60/652,459, 
filed on Feb. 11, 2005, entitled, “New High Performance 
Algorithmic Solution for Diagnosis Problems.” 

GOVERNMENT RIGHTS 

The invention described herein was made in the perfor- 
mance of work under a NASA contract, and is subject to the 
provisions of Public Law 96-517(35 USC 202) in which the 
Contractor has elected to retain title. 

FIELD OF INVENTION 

The present invention relates to techniques for solving 
diagnosis problems, and more particularly, to a system and 
method using an algorithmic method for solving diagnosis 
problems. 

BACKGROUND OF INVENTION 

Technological advances have led to the conception of a 
variety of complex devices and systems. When a system 
fails, a technician or user typically fixes the system by 
identifying and replacing (or fixing) the failed component. 
To ease this process, a variety of model-based diagnostic 
tools have been developed that assist a user in identifying the 
failed component. 

A diagnosis problem arises when a system’s actual behav- 
ior fails to match the expected behavior, thereby exhibiting 
symptoms (anomalies). System diagnosis is then the task of 
identifying faulty components that are responsible for 
anomalous behavior. To solve the diagnosis problem, one 
typically finds the minimal set of faulty components that 
explain the observed symptoms. The most disciplined tech- 
nique to diagnosis is termed “model-based” because it 
employs knowledge of devices’ operations and their con- 
nectivities in the form of models. The model-based approach 
reasons from first principles and affords far better diagnostic 
coverage of a system than traditional rule-based diagnosis 
methods, which are based on a collection of specific symp- 
tom-to-suspect rules. 

The diagnosis process starts with identifying symptoms 
that represent inconsistencies (discrepancies) between the 
system’s model (description) and the system’s actual behav- 
ior. Each symptom identifies a set of conflicting components 
(i.e., conflict set) as initial candidates. A minimal diagnosis 
set is the smallest set of components that intersects all 
conflict sets. The underlying general approach in different 
model-based diagnosis approaches can be described as a 
two-step “divide-and-conquer” technique wherein finding 
the minimal diagnosis set is accomplished in two steps: 1 ) 
Generating conflict sets from symptoms; and 2) Calculating 
minimal diagnosis set from the conflict sets. As shown in 
FIG. 1, the conflict generation corresponds to forming a 
collection of conflict sets 100, and calculating the minimal 
diagnosis 102 corresponds to the solution of the Hitting Set 
problem for this collection. The Hitting Set is the solution 
that would explain all of the symptoms. 

However, there are major drawbacks in the current model- 
based diagnosis techniques in efficiently performing the 
above two steps that severely limit their practical application 
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to many systems of interest. First, existing conflict-gener- 
ating algorithms are all based on various versions of the 
constraint propagation method and truth maintenance sys- 
tems. The problem with these methods is that not only do 
5 they need exponential time, but they also require exponen- 
tial memory to be implemented. Therefore, these methods 
cannot handle realistic systems with a large number of 
components. Second, in order to find the minimal diagnosis 
set, current model-based diagnosis techniques rely on algo- 
111 rithms with exponential computational costs and thus, are 
highly impractical for application to many systems of inter- 
est. 

Thus, a continuing need exists for a system that allows a 
15 user to efficiently identify a minimal diagnosis set for 
complex systems with a large number of components. 

SUMMARY OF INVENTION 

20 The present invention relates to a method for calculating 
a minimal Hitting Set from a collection of sets. The method 
includes an act of using a computer system to analyze a 
system having a collection of sets to derive the minimal 
Hitting Set from the collection of sets. The computer system 
25 is configured to perform multiple acts to determine the 
minimal Hitting Set. For example, the computer system is 
configured to perform acts of receiving information regard- 
ing the collection of sets from the system; creating an 
incidence matrix of the collection of sets; creating nodes 
30 from the incidence matrix such that each node is a node in 
a search tree and has a label that includes a sub-matrix of the 
incidence matrix, and two disjoint subsets of the columns of 
the incidence matrix; calculating an upper bound and cal- 
culating a lower bound for each label, where the upper 
35 bound and lower bound define a range of solutions for the 
minimal Hitting Set; and determining whether each label has 
any child nodes or is a leaf node without any child nodes. If 
the label does not have any child nodes, then stopping the 
process and designating the label as a leaf node. If the label 
40 does have a child node, then splitting the label to create two 
new labels. Finally, repeating acts of calculating and deter- 
mining until all labels have been designated as leaf nodes, 
with information gathered from leaf nodes determining a 
solution of the Hitting Set, thereby allowing a user to 
45 identify the Hitting Set. 

In the act of calculating the minimal Hitting Set, the 
minimal Hitting Set is a minimal diagnosis set and the 
collection of sets is a collection of conflict sets. In this 
aspect, the method for calculating a mini mal Hitting Set 
50 calculates a solution for a diagnosis problem. 

In the act of creating an incidence matrix, the incidence 
matrix is created as a binary matrix denoted as mxn binary 
matrix A having columns and rows. The columns of matrix 
A are labeled by numbers 1,2,... ,n, and any subset of these 
columns are denoted as a subset of {1,2, . . . ,n}. 

In the act of creating nodes, each node has a label (X) in 
a form of 7.=(M, T ( „, T out ). Additionally, M is a sub-matrix 
of matrix A, and T,„ and T out are disjoint subsets of the 
60 columns of matrix A. T I>( denotes a set of columns of matrix 
A considered as part of the minimal diagnosis set. Alterna- 
tively, T out denotes a set of columns of matrix A that are 
considered not as part of the minimal diagnosis set. 

The act of calculating an upper bound further comprises 
65 acts of calculating a value for the upper bound of A.=(M, T,„, 
T oM ) and a calculating an upper bound set, by perfonning 
the following acts: 
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Initializing the value of the upper bound as U=size of T,„ 
and initializing the upper bound set as the set T in . The 
size of T,„ is the number of elements in T,„; 

Choosing a column a 1 of M with a maximum weight. The 
column with the maximum weight is the column of M 5 
with the largest number of 1 ’s; 

Increasing the value of U by 1 and adding the column a : 
to the upper bound set; 

Constructing the matrix from M by deleting the 
column a! and all rows of M that correspond with to 
non-zero components of the column a, . If the matrix 
is empty then the current value of U is the upper 
bound and a solution for the Hitting Set is the upper 
bound set; otherwise 

Repeating the acts of choosing through repeating on the 15 
matrix Mj. 

In the act of calculating a lower bound, a value for the 
lower bound of X=(M, T in , T out ) is equal to the size of T,„ 
plus k/L. k denotes the number of rows of the matrix M and 
L denotes the maximum weight of the columns of M. 20 

In the act of determining whether each label has any child 
nodes or is a leaf node without any child nodes, a label is 
determined to be a leaf node if any of the following are true: 

if T,„UT olrt ={l,2, . . . ,n}, where U denotes the union of 
two sets; 25 

if the value for the lower bound of X s the current value for 
the upper bound; 

if the columns of T jn form a solution such that if the 
columns in T,„ are added together to form a resulting 
vector, then the resulting vector has no zero compo- 30 
nent; or 

if M contains an all-zero row. 

In another aspect, the act of splitting the label to create 
two new labels further comprises an act of splitting label 
X=(M, T t „, T out ) into label k 0 and label X r Label X 0 is 35 
defined as: 

X 0 =(Remove_0 [M, j], T,.„, T„„U{j}). 

j denotes a column of M with a maximum weight and J is the 
corresponding column in the original matrix. Remove^O 
[M, j] denotes a sub-matrix of M obtained by deleting the j' 7 ' 
column of M. Label X : is defined as: 

X 1 =(Remove_l [M, j], T ( „U{j}- TJ- 

Additionally, j denotes a column of M with a maximum 
weight and j is the corresponding column in the original 

matrix. Remove 1 [M, j] denotes a sub-matrix of M 

obtained by deleting the j"' column of M and deleting all 
rows of M that correspond with non-zero components of that 
column. 

In yet another aspect, the act of splitting a label fiirther 
comprises acts of: 

on label X=(M, T in , T out ), if the j' 7 ' column of the matrix 
M is an all-one column, then changing the label X to the 
label ({ }, T !>; U{j }, T our ), where U denotes the union of 
two sets, { } is an empty matrix and j is a corresponding 
column in the original matrix; 

on label X=(M, T,„, T oi „), if the matrix M has a row of 
weight one, with 1 as its j' 7 ' component, then changing 
the label X to the label (Removed_l [M, j], T,„U{J}, 
T out ) where U denotes the union of two sets, J is the 
corresponding column in the original matrix, and 

Remove 1 [M, j] denotes a sub-matrix of M obtained 

by deleting the ] th column of M and deleting all rows of 
M that correspond with non-zero components of that 
column; 


4 

on label X=(M, T,„, T our ), if the matrix M has an all-one 
row, then deleting that row; 

on label X=(M, T ; -„, T our ), if ] ,h column of the matrix M is 
an all -zero column, then changing the label X to the 
label (Remove_0 [M, j], T IU , T ouf U{j}), where U 
denotes the union of two sets, j is the corresponding 

column in the original matrix, and Remove 0 [M, j] 

denotes a sub-matrix of M obtained by deleting the ) ,h 
column of M; 

on label X=(M, T,„, T out ), if the matrix M has two equal 
rows, then deleting one of them; and 
repeating the above acts until all of the above acts are 
exhausted. 

In yet another aspect, the method fiirther comprises an act 
of applying a Branch-and-Bound technique to the incidence 
matrix A to find an optimal solution for the problem, 
comprising acts of: 

initializing a label X 0 as X 0 =Rules*[{A, { }, { }}], where 
Rules* denotes applying and repeating the acts of 
paragraph 23 until all of the acts of paragraph 23 are 
exhausted; 

initializing a set of labels as {X 0 }; 
initializing a value of the upper bound U as the upper 
bound of X 0 ; 

initializing a solution set as the upper bound set of X 0 ; 
choosing a label X from the set of labels; 
deleting X from the set of labels; 
computing Uj as the upper bound of X and its correspond- 
ing upper bound set; 

if Uj <U, then changing the value of U to Uj and the value 
of the solution set to the upper bound set; 
if the label X is not a leaf, then splitting X to labels X 0 and 
Xj and adding the new labels X 0 and X : to the set of 
labels; 

if the set of labels is not empty, repeating the acts of 
choosing a label X through the act of repeating the acts; 
if the set of labels is empty, then the outputting is the 
solution set and stopping. 

In another aspect, the present invention also includes a 
system that is configured perform the acts above. 

Finally, as can be appreciated by one skilled in the art, the 
present invention also includes a computer program product. 
The computer program product includes computer-readable 
instruction means stored on a computer-readable medium 
for causing a computer to perform the operations and acts 
described herein. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The objects, features and advantages of the present inven- 
tion will be apparent from the following detailed descrip- 
tions of the various aspects of the invention in conjunction 
with reference to the following drawings, where: 

FIG. 1 is an illustration of an exemplary diagnosis of a 
Hitting Set of conflict sets; 

FIG. 2 is a block diagram depicting the components of a 
diagnosis system according to the present invention; 

FIG. 3 is an illustrative diagram of a computer program 
product embodying the present invention; 

FIG. 4 is an illustration of a matrix, depicting a Hitting Set 
Problem being mapped onto Integer Programming; 

FIG. 5 is a table illustrating exemplary test results of the 
present invention; 

FIG. 6 illustrates an exemplary recursive procedure of 
pseudo-code for computing an upper bound; 
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FIG. 7 is a graph illustrating typical solution sets of the 
Linear Programming (LP) relaxation and Integer Program- 
ming (IP) problems; 

FIG. 8 is a table illustrating the performance results of the 
algorithm of the present invention and its comparison with 
the traditional Branch-and-Bound method; 

FIG. 9 is an illustration of an exemplary search tree; 

FIG. 10 is a graph illustrating an exemplary vertex cover 
as applied to a Vertex Covering Problem; and 

FIG. 11 is a graph illustrating the perfonnance of the 
algorithms of the present invention contrasted with the 
traditional technique for solving a Vertex Covering Problem. 

DETAILED DESCRIPTION 

The present invention relates to a solution for diagnosis 
problems, and more particularly, to a system and method 
using an algorithmic solution for diagnosis problems. The 
following description is presented to enable one of ordinary 
skill in the art to make and use the invention and to 
incorporate it in the context of particular applications. Vari- 
ous modifications, as well as a variety of uses in different 
applications will be readily apparent to those skilled in the 
art, and the general principles defined herein may be applied 
to a wide range of embodiments. Thus, the present invention 
is not intended to be limited to the embodiments presented, 
but is to be accorded the widest scope consistent with the 
principles and novel features disclosed herein. 

In the following detailed description, numerous specific 
details are set forth in order to provide a more thorough 
understanding of the present invention. However, it will be 
apparent to one skilled in the art that the present invention 
may be practiced without necessarily being limited to these 
specific details. In other instances, well-known structures 
and devices are shown in block diagram form, rather than in 
detail, in order to avoid obscuring the present invention. 

The reader’s attention is directed to all papers and docu- 
ments which are filed concurrently with this specification 
and which are open to public inspection with this specifi- 
cation, and the contents of all such papers and documents are 
incorporated herein by reference. All the features disclosed 
in this specification, (including any accompanying claims, 
abstract, and drawings) may be replaced by alternative 
features serving the same, equivalent or similar purpose, 
unless expressly stated otherwise. Thus, unless expressly 
stated otherwise, each feature disclosed is one example only 
of a generic series of equivalent or similar features. 

Furthermore, any element in a claim that does not explic- 
itly state “means for” performing a specified function, or 
“step for” performing a specific fimction, is not to be 
interpreted as a “means” or “step” clause as specified in 35 
U.S.C. Section 112, Paragraph 6. In particular, the use of 
“step of’ or “act of’ in the claims herein is not intended to 
invoke the provisions of 35 U.S.C. 112, Paragraph 6. 

Before describing the invention in detail, first a glossary 
of terms used in the description and claims is provide. Next, 
a description of various principal aspects of the present 
invention is provided. Subsequently, an introduction is pro- 
vided to provide the reader with a general understanding of 
the present invention. Finally, the details of various aspects 
of the present invention are provided to give an understand- 
ing of the specific details. 

(1) Glossary 

Before describing the specific details of the present inven- 
tion, a glossary is provided in which various tenns used 
herein and in the claims are defined. The glossary provided 
is intended to provide the reader with a general understand- 
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ing of the intended meaning of the tenns, but is not intended 
to convey the entire scope of each term. Rather, the glossary 
is intended to supplement the rest of the specification in 
more accurately explaining the terms used. 

5 Conflict Set — The tenn “conflict set” as used with respect 

to the present invention refers to a set of components of the 
system such that the assumption of non-faultiness of all of 
them is not consistent with the model of the system and 
observations. 

to Hitting Set — The tenn “Hitting Set” as used with respect 
to the present invention, refers to a minimum set that 
intersects all conflict sets. 

Incidence Matrix — The term “incidence matrix” as used 
with respect to the present invention refers to a 0/1 matrix 
15 whose rows are labeled by the conflict sets {C 1: C 2 , . . . , C„,} 
and columns are labeled by the components of the systems 
{nij, m 2 , . . . , ni„}, and the entry ay=l if the ] ,h component 
m- belongs to the \ Th conflict set C,-, otherwise a y =0. 

Instruction Means — The term “instruction means” as used 
20 with respect to this invention generally indicates a set of 
operations to be performed on a computer, and may repre- 
sent pieces of a whole program or individual, separable, 
software modules. Non-limiting examples of “instruction 
means” include computer program code (source or object 
25 code) and “hard-coded” electronics (i.e., computer opera- 
tions coded into a computer chip). The “instruction means” 
may be stored in the memory of a computer or on a 
computer-readable medium such as a floppy disk, a CD- 
ROM, and a flash drive. 

30 (2) Principal Aspects 

The present invention has three “principal” aspects. The 
first is a system for solving diagnosis problems. The system 
is typically in the form of a computer system operating 
software or in the form of a “hard-coded” instruction set. 
35 This system may be incorporated into a wide variety of 
devices that provide different functionalities. The second 
principal aspect is a method, typically in the form of 
software, operated using a data processing system (com- 
puter). The third principal aspect is a computer program 
40 product. The computer program product generally repre- 
sents computer-readable instructions stored on a computer- 
readable medium such as an optical storage device, e.g., a 
compact disc (CD) or digital versatile disc (DVD), or a 
magnetic storage device such as a floppy disk or magnetic 
45 tape. Other, non-limiting examples of computer-readable 
media include hard disks, read-only memory (ROM), and 
flash-type memories. These aspects will be described in 
more detail below. 

A block diagram depicting the components of a diagnosis 
50 system of the present invention is provided in FIG. 2. The 
diagnosis system 200 comprises an input 202 for receiving 
information from at least one component for use in detecting 
a conflict set. Note that the input 202 may include multiple 
“ports.” Typically, input is received from at least one com- 
55 ponent, a non-limiting example of which includes a micro- 
computer within a larger, computer operated system. An 
output 204 is connected with the processor for providing 
information regarding the conflict set and diagnostic solu- 
tion to the conflict set. Output may also be provided to other 
60 devices or other programs; e.g., to other software modules, 
for use therein. The input 202 and the output 204 are both 
coupled with a processor 206, which may be a general- 
purpose computer processor or a specialized processor 
designed specifically for use with the present invention. The 
65 processor 206 is coupled with a memory 208 to pennit 
storage of data and software that are to be manipulated by 
commands to the processor 206. 
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An illustrative diagram of a computer program product 
embodying the present invention is depicted in FIG. 3. The 
computer program product 300 is depicted as an optical disk 
such as a CD or DVD. Flowever, as mentioned previously, 
the computer program product generally represents com- 
puter-readable instructions stored on any compatible com- 
puter-readable medium. 

(3) Introduction 

The diagnosis problem arises when a system’s actual 
behavior fails to match the expected behavior, thereby 
exhibiting symptoms (anomalies). System diagnosis is then 
the task of identifying faulty components that are respon- 
sible for anomalous behavior. To solve the diagnosis prob- 
lem, one must find the minimal set of faulty components that 
explain the observed symptoms (i.e., minimal diagnosis set). 

The present invention relates to a system for solving 
diagnosis problems by determining the lower and upper 
bounds on the size of the solution. The system uses these 
bounds to generate a solution window for the problem. The 
present invention also includes a Branch-and-Bound tech- 
nique that is faster than existing techniques in terms of 
number of operations (by exploiting the structure of the 
problem). Using the concept of a solution window, the 
Branch-and-Bound technique allows a massive reduction 
(pruning) in the number of branches. Furthermore, as the 
Branch-and-Bound proceeds, the solution window is 
dynamically updated and narrowed to enable further prun- 
ing. 

The present invention includes a powerful, yet simple, 
representation for calculating the minimal diagnosis set. 
This representation enables mapping onto a 0/1 Integer 
Programming problem. Mapping onto the 0/1 Integer Pro- 
gramming problem enables the use of a variety of algorithms 
that can efficiently solve the problem for up to several 
thousand components. The algorithms of the present inven- 
tion are significantly improved over the existing ones, 
enabling efficient diagnosis of large, complex, systems. In 
addition, this mapping enables a fast determination of the 
lower and upper bounds on the solution (i.e., the minimum 
number of faulty components). 

For further clarification, specific aspects of the present 
invention are described below. Additionally, tests were per- 
formed to further demonstrate the efficacy of the present 
invention. For further illustrative purposes, the results of 
performance tests of the present invention are compared 
with the traditional and standard algorithms. As shown 
below, the test results demonstrate that the present invention 
has a performance with more than ten times improvement 
over the traditional approach. 

(4) Details of Specific Aspects 

For clarity, various details of specific aspects of the 
present invention will be described in separate sub-sections 
below. The first section describes an algorithmic approach to 
the diagnosis problem. The second section describes the 
bounds on the diagnosis. The third section provides an 
introduction to the Branch-and-Bound method. The fourth 
section describes a Branch-and-Bound method according to 
the present invention. The fifth section describes perfor- 
mance results of the method of the present invention con- 
trasted with a traditional Branch-and-Bound method. The 
sixth section explains mapping other problems into the 
Integer Programming Problem. And finally, the seventh 
section includes a brief summary of the present invention. 

(4.1) Algorithmic Approach to the Diagnosis Problem 

To overcome the complexify of calculating a minimal 
diagnosis set, the present invention relates the calculation 
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and solution of the Hitting Set problem to the solution of 
Integer Progra mmi ng problems. 

The Hitting Set Problem, also known as the Transversal 
Problem, is one of the key problems in the combinatorics of 
5 finite sets and the theory of diagnosis. The problem is 
described as follows. A collection S={S 1 , . . . , S m } of 
nonempty subsets of a set M is given. A Hitting Set of S is 
a subset H of M that intersects every set in the collection S. 
Of course, there are always trivial Hitting Sets. For example, 
to the background set M is always a Hitting Set. However, 
interest remains in minimal Hitting Sets with minimal 
cardinality. A Hitting Set H is minimal if no proper subset of 
H is a Hitting Set. 

FIG. 4 illustrates the mapping of the Hitting Set Problem 
15 onto Integer Programming. AO/1 (binary) matrix A 400 is 
defined as the incidence matrix of the collection of the 
conflict sets. In other words, the entry a„ 402=1 if and only 
if the j' ; ' element m ; 404 belongs to the \‘ h set C, 406. 
Additionally, x=(x 15 x 2 , . . . , x„) is a binary vector. x =l if 
20 the element my 404 belongs to the minimal Hitting Set and 
hence the minimal diagnosis set, otherwise x 2 =0. Thus, the 
Hitting Set Problem is shown as a 0/1 Integer Programming 
Problem in the following formulation: 

2^ minimize x 1 +x 2 + . . . +x„, 

subject to Ax^b, x y =0, 1, (1) 

where b r =(l, ... ,1) is a vector whose components are all 
equal to one. This mapping provides for the utilization of 
3 ' existing Integer Programming algorithms, such as commer- 
cially available Integer Programming tools. 

(4.2) Bounds on the Diagnosis 

The mapping of the present invention offers two addi- 
35 tional advantages that can be exploited to develop yet more 
efficient algorithms. First, the mapping represents a special 
case of the Integer Programming Problem due to the struc- 
ture of matrix A (i.e., a binary matrix) and vector b. Second, 
by using this mapping, the minimum number of faulty 
40 components can be determined without solving the problem 
explicitly. For this purpose, the 1-norm and 2-norm of 
vectors are defined as: 


NIi = 2 W- IWh = ■ 


Using the above equations and vector b from the optimiza- 
tion problem (1), results in ||b|| 1 =m and ||b|| 2 =Vm. Because the 
elements of both vectors Ax and b are positive, the following 
two inequalities can be derived: 

55 MiXlMli ^ m => IWIj am/IIAIIj t (2) 

IHII2 X IMF £ V/7 => iwi 2 2 V7T/ ||A|| 2 J ’ 

60 where || denotes the norm, x denotes multiplication, and/ 
denotes division. Since x is a binary vector, then both norms 
in equation (2) give the bound on the size of the solution 
(i.e., the number of nonzero elements of vector x). The 
number of nonzero elements of vector x corresponds to the 

65 minimal diagnosis set. It should be noted that depending on 
the structure of the problem (i.e., the 1- and 2-norm of the 
matrix A and m), a sharper bound can be derived from either 
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of the norms in equation (2). The present invention derives 
the bounds on the solution of the problem without explicitly 
solving the problem. Such a priori knowledge on the size of 
solution can be used for developing much more efficient 
algorithms for the problem. 5 

Furthermore, an upper bound for the solution size can be 
found using monotonicity of the Integer Programming of 
optimization problem (1). Monotonicity means that if x is a 
solution of Ax=b and y=x, then y is also a solution of the 
same system. Note that finding a 0/1 solution x for the to 
system Ax =b is equivalent to finding a subset of the 
columns of the matrix A, such that their sum is a vector with 
components all equal to or greater than 1 . Any such solution 
provides an upper bound for the optimization problem ( 1 ), 
since that problem solves a minimal set of such columns. 15 

Therefore, to find an upper bound, a column C of A with 
the largest weight must first be chosen. Then, a submatrix of 
A is constructed by deleting the column C : and all rows of 
A that correspond to non-zero components of C 1 . The same 
process is then applied to the new matrix (i.e., submatrix of 20 
A), until an empty matrix is derived. The columns C 1; 

C 2 , . . . , C t that are obtained determine a solution for AxSb 
and the number t is an upper bound for the solution of the 
Integer Programming problem of optimization problem (1 ). 
FIG. 5 is a table illustrating test results of the present 25 
invention, indicating that the upper bound is actually sharp, 
particularly for a small size solution. Note that it is easy to 
modify this algorithm in a way that it also provides a vector 
a, such that the vector Aa realizes the corresponding upper 
bound. 30 

FIG. 6 illustrates an exemplary recursive procedure of 
pseudo-code for computing the upper bound. As can be 
appreciated by one skilled in the art, the pseudo-code shown 
in FIG. 6 is a non-limiting example of pseudo-code operable 
for computing the upper bound, and that the pseudo-code 35 
can be altered to provide the same or similar result. 

There are two simple rules that assist the above algorithm 
in the following two extreme cases. The algorithm will 
likely end up with sub-matrices that these rules can be 
applied to. The rules are as follows: 40 

1. If the matrix A has an all -one column, then the upper 
bound is equal to 1 ; and 

2. If some row of the matrix A has weight 1 , then remove 
that row and the corresponding column to obtain the matrix 
Aj and Upper_Bound [A]=l+Upper_Bound [AJ, where 45 
Upper_Bound [AJ and Upper_Bound [A] denote the values 
of the upper bound for the corresponding matrices. 

The upper bound can also be improved by a step-by-step 
method and in an iterative fashion. For example, the cost of 
the k“ step in the iteration is of the order of n 4 (i.e., CXn 4 )), 50 
where n is the number of the columns of the matrix. In this 
example, the first few steps are practically efficient. More 
specifically, for a fixed k, instead of choosing the maximum 
weight column for the vector a 1 , the sum of k columns of A 
can be chosen, with all possible such vectors thereafter being 55 
tried. 

As another application of the a priori lower bound, the 
cases where a high number of faulty components requires 
another course of action instead of usual identification of 
faulty components, can be separated. These cases can be 60 
separated before starting to solve the hard problem of 
finding the minimal Flitting Sets. Also a good lower and 
upper bound can determine whether the enhanced brute- 
force algorithm can provide a solution efficiently. As it was 
stated before, this algorithm has a complexity of 0(n r ), 65 
where t is the number of faulty components and n is the 
number of the columns of the matrix. 
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(4.3) Introduction to the Branch-and-Bound Method 

The Branch-and-Bound method is one of the most com- 
mon methods for solving intractable problems. In the case of 
the Integer Programming (IP) problem, this method tradi- 
tionally begins by solving the Linear Programming (LP) 
relaxation of the IP (i.e., by removing the condition that the 
variables x y , in the optimization problem (1) of paragraph 
00066, are integers). 

FIG. 7 is a graph 700 illustrating typical solution sets of 
the LP relaxation and IP problems. As shown in FIG. 7, the 
polygon 702 represents the solution set of the LP relaxation 
and the grid points 704 inside tins polygon 702 represent the 
solution set of the IP problem. 

If the optimal solution of the LP relaxation consists only 
of integer values, then the optimal solution of the LP 
relaxation will be the optimal solution for the IP problem. 

Otherwise, if the IP problem is defined by a system such 
as the optimization problem (1), the optimal solution of the 
LP relaxation provides a lower bound for the IP problem. In 
such a case, one of the non-integer values of the optimal 
solution of the LP relaxation can be chosen (e.g., x =a), with 
two new sub-problems being defined by adding the condi- 
tions x^.gfa] and x^SJaj+l to the system, where [a] denotes 
the integer part of a. Then, the two new sub-problems need 
to be solved. By continuing this procedure, the sub-problems 
of the original IP problem are defined. Once an integer 
optimal solution for the LP relaxation of a sub-problem is 
found, that solution gives us an upper bound for the IP 
problem. After finding such an integer solution, any sub- 
problem is eliminated whose (LP relaxation) lower bound is 
larger than the upper bound provided by some other sub- 
problem. This procedure is continued until all sub-problems 
are eliminated or an integer optimal solution is found for the 
corresponding LP relaxation problem. At the end, the opti- 
mal solution for the IP problem is the best of the optimal 
integer solutions of the sub-problems. 

The above procedure can be generalized as follows. To 
begin the Branch-and-Bound method, the following acts 
need to be performed: 

Partitioning a problem P into a collection P 1; P 2 , . . . , P A 
of mutually disjoint sub-problems; 

Finding a lower bound for each sub-problem P,; 

Finding an integer solution for a subclass of sub-problems 
(in this case by obtaining an upper bound); and 

Optionally, determining whether the integer solution is 
optimal for the corresponding sub-problem. 

Note that in the case of LP relaxation, whenever an integer 
solution for a sub-problem is found, it is guaranteed that it 
is an optimal solution for that sub-problem. Starting with the 
original problem P, defined by a system like the optimization 
problem (1), the M' 7 ' step of the procedure provides parti- 
tions P m ,1) P m ,25 • • • J t of mutually disjoint sub-problems. 

For each sub-problem P , a lower bound L m ■ is found and, 
if possible, an integer solution and corresponding upper 
bound U . is found. Then, a decision is made as to which 
sub-problems P . should be eliminated at tins step. There 
are two criteria for this decision: 

1 . The lower bound L ■ is larger than upper bound U m>A 
of some other sub-problem P m i ; and 

2. .An optimal solution for the sub-problem P . is found. 

In the case of (2) (i.e., optimal solution is found), the 

record of the best optimal solution of the sub-problems is 
kept. Then, the act of partitioning a problem P into a 
collection of mutually disjoint sub-problems is applied to the 
remaining sub-problems and to find the lower and (if pos- 
sible) upper bounds for the new sub-problems. This proce- 
dure is continued until no sub-problem remains. Then, the 
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best of the optimal solutions of sub-problems is the optimal 
solution for the original problem. 

(4.4) Branch-and-Bound Method of the Present Invention 
The Branch-and-Bound algorithm of the present inven- 
tion is based on the above methods for computing lower and 5 
upper bounds for diagnosis. Hie monotonicity property of 
the Integer Programming problem is also exploited. The 
rational behind this approach and its possible advantages 
over the standard Branch-and-Bound method, are as fol- 
lows: 10 

Since the optimal solution is one of the points on the 
discrete grid of FIG. 7, the relaxation phase of the 
present invention applies directly to the discrete grid. 
Alternatively, the standard method starts with a much 
larger set that consists of not only the discrete grid but 15 
also all real points inside the polygon. 

Using the present invention, for each sub-problem, a 
lower bound in linear time is found. Conversely, the 
time of LP relaxation of the standard method is 0(n 3 ). 

For each sub-problem, an upper bound can be found using 
the present invention. Alternatively, using the standard 
method, the upper bound is found only in cases where 
the LP relaxation of the sub-problem at hand ends up 
with an integer solution. Thus, the present invention 2 $ 
provides a greater chance of eliminating sub-problems 
with large lower bounds. 

Before describing the method according to the present 
invention, a set of useful functions and notation are intro- 
duced. 30 

Matrix A is an mxn binary matrix. The columns of A are 
labeled by the numbers 1 , 2 , . . . , n, and any subset of these 
columns is denoted as a subset of {1, 2, ... , n}. Similar to 
the traditional Branch-and-Bound method, the algorithm of 
the present invention is also based on a search of the nodes 35 
of a tree. Each node of the search tree has a label of the form: 

(M, T,.„, T„J, 

where M is a submatrix of the original matrix A, and T„, and 
T out are disjoint subsets of the columns of the original matrix 
A. The meaning of this partition is that T ( „ is the set of the 
columns (of the original matrix A) considered as part of the 
solution, while T out is the set of the columns (of the original 
matrix A) that are considered not to be part of the solution. 
Note that a solution of the optimization problem (1) can be 
considered as a subset of the columns of matrix A, whose 
addition is a vector with all non-zero components. 

Following is a list of auxiliary functions and sub-routines 
that are used in implementing the method of the present S(| 
invention. As can be appreciated by one skilled in the art, the 
titles and names are for illustrative purposes only and can be 
changed provided the operation provides the same or a 
similar result. 

(4.4.1) Function Place Finding 55 

Consider a node with the label (M, T ( „, T out ). Since M is 
a submatrix of the original matrix A, every column of M 
corresponds with a unique column of A. As a non-limiting 
example, the column 1 of M corresponds with the column 3 
of A and the column 2 of M corresponds with the column 7 60 
of A, and so on. Therefore, using a placing finding function, 
the label of a column of M in A can be referred to without 
any ambiguity. In fact, given the sets T I>; and T ouP for every 
column j of M, it is possible to find the corresponding 
column in the original matrix A. For illustrative purposes, 65 
this relation is denoted by the function Place_Finding. 
Additionally, to keep the notation simple, instead of writing 
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Place_Finding[T„„ T our> j], j alone may be used if the sets T„, 
and T out are understood from the context. 

(4.4.2) Function Remove 0 and Remove 1 
As described in further detail below, a label can be split 
to create two new labels. For example, label X=(M, T jn , T out ) 
can be split into label Xq and label X x . Label X 0 is defined as: 

^(Remove — 0 [M, j], T,.„, T„„,U{]}), 


where j denotes a column of M with a maximum weight, U 
denotes the union of two sets, and j is the corresponding 
column in the original matrix. As applied to this section, the 
function Remove__0 [M, j] denotes a sub-matrix of M 
obtained by deleting the j rh column of M. 

Additionally, label Xj is defined as: 

N-(Remove„l [M, j], T,„U{i}- T„J, 

where the function Remove 1 [M, j] denotes a sub-matrix 

of M obtained by deleting the ] ,h column of M and deleting 
all rows of M that correspond with non- zero components of 
that column. 

As a non-limiting example, consider the following matrix: 


0 0 1 l 
110 1 
10 10 
,0101, 


Then, to compute Remove__l [M, 2], the 2 nd column of M 
is deleted. Since the 2 nd and 4 th elements (e.g., components) 
of this column are 1 , the 2" d and 4 th rows are deleted as well. 
The result is as follows: 


Remove_l [M , 2] = 


o l n 

110/ 


For the function Remove__0 [M, 2], only the 2 nd column 
of M is deleted. The result is as follows: 


( o i n 


Remove_0[M, 2] = 


1 0 1 
1 1 0 


10 0 lj 


(4.4.3) Function Rule* 

This function is based on some rules that simplify the 
process of finding the optimal solution for the optimization 
problem ( 1 ): 

Rule 1 If matrix A has an all-one column C -, then the set 
consisting of the ]‘ h element x alone is the Hitting Set; 
or the vector x with 1 at its j' 7 ' component and zero at 
other components, is an optimal solution for the opti- 
mization problem ( 1 ). 

Rule 2 If matrix A has a row of weight one, with 1 as its 
] th component, then the j* element Xj is contained in the 
minimal Hitting Set. In this case, the optimization 
problem ( 1 ) can be simplified by removing this row and 
the j'* column, with the ]‘ h element x ; being added to the 
solution of the new system. 

Rule 3 If matrix A has an all-one row, delete that row. 
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Rule 4 If matrix A has an all-zero column, delete that 
column. 

Rule 5 If matrix A has two equal rows, delete one of them. 

Following is a description of how the above Rules affect 
the labels of the nodes in the search tree, with the label of 5 
a node being denoted as (M, {T,„, T oi , r })- 

First, the action of Rule 1 is described. If the matrix M 
does not have an all-one column (i.e., a column having 
elements all equal to “1”), then no action is performed and 
the label is unchanged. Otherwise, assume that the j* to 
column is all-one. In such a case, Rule 1 changes the label 
to the following: 

(0, T,.„U{j}, T„„), 

where 0 denotes an empty matrix and U denotes the union 
of two sets (i.e., the union of l in and j (the corresponding 
column in the original matrix)). 

In a more formal language, function Rule 1 is defined on 

a set of labels as follows: . 


Rule_l[(M, T m , T„,)} = 


( M , T m , T out ), 

(0. T in U {}'}. T out ), 


25 


otherwise. 

Finally, the following function describes Rule 5: 


Rule_5[(M, T,„, T„,)] = 


(AT, T in , T„ t ), 
(M, T in , T out ), 


if M has two equal rows, 
otherwise, 

where the matrix M' is obtained from M by deleting one of 
the equal rows. 

Note that once one of these rules is applied on a label 
A. i (M, T,„, T our ), and the result is the label X 2 , then it may 
be possible to apply one of these rules to /. 2 . and so on. For 
this reason, the Function Rule* is defined on the set of labels 

as repeated applications of Rules 1-5, until none of the 

rules can be applied anymore. It is easy to show that Rule* 
is well-defined (i.e., the result of Rule*(X) does not depend 
on the order that the functions Rule_l and Rules_2 are 
applied). As a non-limiting example, consider the following 
label: 


if M has no all — one columns, 


if the jth column is an all — one column. 

To define the action of Rule 2, first a useful notation is 
introduced, e - denotes the unit binary vector (of weight one) 
with its only 1 component at ] th position. The action of Rule 
2 can now be described by the following function: 


Rule_2[(M, T;„, T„,i] = 


(Remove_l[M, j], T m (J {)}. Tout), 
[M, T, n , T out % 


35 


0 , 0 ), 

where 

0 0 1 0 1 

0 10 0 0 

0 0 0 1 0 

M] = 1 0 1 1 1 ' 

10 10 0 
, 00111 . 


if M has a row 
equal to e, 
otherwise. 

Hie action of Rule 3 is described by the following 
function: 


(M\ T in , Tout), 

Rule 9 (M, Ti„, r„,)] = 

{M,Ti„,T„ t ), 


The 2 nd row of M is equal to e 2 and its 3 rd row is equal to 
e 4 . Therefore, when applying Rule__2 there are two possible 
choices. First, the 2 nd row is removed, the result is the label 
X 2 (M 2 , {2}, 0), where 


M; = 


0 10 1 ' 

0 0 10 
1111. 
110 0 
.0111. 


if M has all-one rows, 
otherwise, 

55 

where M' is obtained from M by deleting all all-one rows. 

Hie action of Rule 4 is described by the following 
function: 


The 2 nd row of M 2 is equal to e 3 , thus Rule_2 can be 
applied. The result is the label X 3 =(M 3 , {2, 4}, 0), where 



Rule_4[(M, Ti„, T ou ,)} = 


(Remove_0[M. j], Tj n .T out U j /[ ). 
[M, T m , Tn,,,), 


if the f' 


60 Note that the 3 rd column of M 3 is the A ,h column of the 
original matrix M 3 . Finally, Rule_l is applied and the result 
is the label X 4 =(0, {2, 3, 4}, {1, 5}) (again note that the 2 nd 
column of the matrix M 3 corresponds with the 3' d column of 
the original matrix M, ). Therefore, in this non-limiting 
65 example, 


column of M is all-zero, 


Rule*[N]=(0- (2. 3, 4}, (1, 5}). 
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(4.4.4) Function Split 

As described briefly above, a label can be split into two 
new labels. The label is split when it is determined to not be 
a leaf node in the search tree (i.e., Function Test Leaf as 
described in further detail below). A leaf node is a node that 5 
is at an end of a branch in the search tree and has no child 
nodes. When a node is split, it creates at least two children 
nodes (i.e., child nodes). The (partial) function Split is 
defined on the set of the labels, where the value Split [X] is 
a pair (X 0 , XJ of labels. Suppose that X=(M, T l5 T 2 ). If to 
T i nT 2 ={l, . . . n}, i.e., ifTjHTj is the set of all columns of 
the matrix A, then the function Split is not defined. IT 
denotes the intersection of two sets (i.e., the set of elements 
common between two sets). Otherwise, let jG^HTj be a 
column of the original matrix A which is corresponded with 15 
a maximum-weight column of the submatrix M (if there are 
several such columns then the first one is chosen), ^denotes 
a membership relation, where the left-hand side is not a 
member of the set in the right-hand side. Then, two new 
labels are defined by adding the j* column to the sets T t and 20 
T 2 , respectively. More specifically, the two new labels are 
defined as follows: 

X 0 =(Remove_0 [M, j], T„ T 2 U{j}), 

X 1 =(Remove_l [M, j], T 1 U{]}, T 2 ). 

Using the above formulation, the Split function is defined as: 

Split[X]=(Rule* [X 0 ] ,Rule* [X,]). 

(4.4.5) Function Upper Bound j0 

When solving the optimization problem (1), it is help fill 

to identify the range of solutions for the minimum diagnosis 
set. The range of solutions is defined by the upper bound and 
the lower bound. The Upper_Bound function is defined to 
find the number Upper_Bound [A] as an upper bound on the 
solution of the optimization problem (1). This function is 
extended to the set of labels as follows. For the label X=(M, 

T 1; T 2 ), where M is a submatrix of the original matrix A, 
Upper_Bound [X] provides an upper bound for the optimi- 40 
zation problem defined by (1), augmented by the following 
conditions: 

Xy=l, XyGTj, and 

Xj= 0, xjE T 2 . 45 

Applying the above, it can be shown that Upper_Bound 
[X]=IT 1 l+Upper_Bound [M], 

In the special case that M=0, Upper-Bound [X]=IT 1 I. Note 
that the function Upper_Bound is applied to both matrices 50 
and labels. 

(4.4.6) Function Upper Bound Set 

For the label X=(M, T 1? T 2 ), the function Upper_Bound_ 

Set [X] returns the set which realizes the bound Upper_ 55 
Bound [X] (i.e., the union of Tj and the set of columns of 
matrix M which provide the bound Upper_Bound [M]). 

(4.4.7) Function Lower Bound 

Like the previous function, the lower bound defined by (2) 
is extended to the set of labels. More specifically, for the 60 
label X=(M, T x , T 2 ), where M is a kxj submatrix of the 
original matrix A, Lower_Bound [X]=IT 1 l+k/||M|| 1 . In other 
words, a value for the lower bound of X=(M, T ;>J , T OJ „) is 
equal to the size of T,„ plus k/L, where k is the number of 
rows of the matrix M and L is the maximum weight of the 65 
columns of M. That is, each column of M has a weight and 
the maximum weight is the largest among these numbers. 
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(4.4.8) Function Test Solution 

This function is defined on the set of the labels and its 
value is either True or False. The value of Test_Solution [(M, 
T 1; T 2 )] is True if the columns in the set Tj form a solution 
for the optimization problem (1). Otherwise, the value of the 
function is False. 

(4.4.9) Function Test Leaf 

This function is defined on the set of the labels and its 
value is either True or False. Function Test_Leaf is used to 
determine whether or not a node in the search tree is a leaf 
(i.e., whether that node has any children or not). The 
arguments of this function include a label X=(M, T 1; T 2 ) and 
a value U for the upper bound on the solution of the problem. 
The function is as follows: 


Test_Leaf[A, U ] = 


True if T\ U T 2 = {1> 2, ... , n), or 
True if Lower_Bound[A] s U , or 
True if Test_Solution[A] = True, or 
True if M contains an all- zero row, or 
False otherwise. 


(4.4.8) The Branch-and-Bound Algorithm of the Present 
Invention 

The present invention uses the above functions and ter- 
minology to create a Branch-and-Bound algorithm. The 
algorithm is described below for matrix A: /* solves the 
Hitting Set problem defined by the problem (1)*/ 


1. Labels <-{Rule* [(A, 0, 0)]} 

2. U co /* upper bound */ 

3. Solution <—0 

4. while Labels * 0 

5. chose A = (M, T 1? T 2 ) GLabels 

6. Labels Labels - {A} 

7. If Test_Solution [A] = True & Upper_Bound [A] < U then 

8. Solution <— Tj 

9. U Upper_Bound [A] 

10. end if 

11. If Test_Solution [A] = True & Upper_Bound [A] = U & 
Solution=0 then 

Solution <— Tj 

12. If Upper_Bound [A] < U then 

13. U <— Upper Bound [A] 

14. Solution <— Upper_Bound_Set[A] 

15. end if 

16. If Test Leaf [A, U] = False then 

17. (Aq, A0 < — Split [A] 

18. Labels <— Labels U {A^ A x } 

19. end if 

20. If Test_Leaf [A, U] = True & Upper_Bound [A] = U & 
Solution=0 

21. then Solution Upper_Bound Set [A] 

22. end while 

23. return Solution 


For clarity, the following key can be used to define 
various functions and terminology used in the above algo- 
rithm: 

Labels denotes the set of labels; 

■s— denotes the substitution operation; the right-hand side 
is the new value of the left-hand side; 

0 denotes the empty set (null set); 

U denotes the current best value of the upper bound; 

oo denotes the infinity; 

Solution denotes the solution of the problem and the final 
output; 

/* is a symbol that indicates that the sentence between a 
pair of “/* */” is a comment; and 
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fc denotes a membership relation, where the left-hand side 
is a member of the set in the right-hand side. 

As can be appreciated by one skilled in the art, the above 
terminology associated with each function is used for illus- 
trative purposes only and can be changed provided the 
function provides the same or a similar result. Additionally, 
the above algorithm is a non-limiting example of pseud-code 
for achieving a solution and can be altered to provide the 
same result. 

(4.5) Performance Results of the Present Invention Con- 
trasted with a Traditional Branch-and-Bomid Method 
FIG. 8 is a table illustrating the performance results of the 
algorithm of the present invention and its comparison with 
the traditional Branch-and-Bound method. A tool used for 
the traditional algorithm is the GLPK (GNU Linear Pro- 
gramming Kit), version 3.2. This is a set of routines in the 
ANSI C programming language. The Integer Programming 
routine of GLPK applies a variant of the Branch-and-Bound 
method for the problem of the present invention. The results 
shown in FIG. 8 illustrate the average time and the number 
of iterations (i.e., the number of nodes in the search tree) 
used by these algorithms on 100 random binary matrices. 

For further illustration, consider the following 10x12 
exemplary matrix: 

'0 0000100010 O' 
000010000001 
000110001011 
0 01000100010 
0 0 1 1 1 0 0 1 1 1 1 0 
Ao ” 010010011010 ' 
010101101000 
100101111000 
101000010010 
. 111000010101 , 


-continued 

/0 000010010 0) 
00001000001 
Removed [A 0 , 8] = 00011001011. 

00100010010 
.0 1 0 1 0 1 1 1 0 0 0, 

Note that none of the Rules 1 -5 can be applied on the 

matrix Remove 0 [Aq, 8] . Additionally, only Rule 4 can be 

applied to Remove 1 [A,,, 8] because the 1" column of this 

matrix is an all -zero vector. Applying this rule results in the 
15 following matrix: 

0 0 0 0 1 0 0 1 0 0 ) 

0001000001 
A; =0011001011. 

0 1 0 0 0 1 0 0 1 0 
. 1010111000 . 

Therefore, the labels of the children of Node 0 are 

^,=(A„ 0, {8}). and 

X 2 =(A 2 . {8}. {1}). 

(4.5.2) Node 1 

The label for Node 1 is X^ The updated lower and upper 
bounds are Lower_Bound=2 and U=Upper_Bound=4. The 
upper bound is realized with the set {3, 5, 6, 9}, and 
Test_Leaf [X 1; U]=False. Thus, Node 1 has two children 
(i.e.. Node 3 and Node 4). Column 8 of the matrix A 15 which 
is the 9 th column of the original matrix A 0 , has the maximum 
weight. Then, 


The nodes of the search tree and their labels are as 
follows. 

(4.5.1) Node 0 

For Node 0, the label of the root is X 0 =(A„, 0, 0). In this 
case, apply Rule*[X 0 ]=X 0 . The initial lower and upper 
bounds are Lower_Bound=2, and U=Upper_Bound=5, 
where the upper bound is realized with the set {3, 4, 5, 6, 8}. 
Moreover, Test_Leaf [X 0 , U]=False, thus this node has two 
children (i.e., Node 1 and Node 2). To find the two children, 
notice that column 8 has a maximum weight, then; 


•o 0 
0 0 
0 0 
0 0 
0 0 

Remove_0[/4/, 8] = ^ ^ 

0 1 
1 0 
1 0 
, 1 1 


0 0 0 1 0 1 0 0 
0 0 1 0 0 0 0 1 
0 1 1 0 0 0 1 1 
1 0 0 0 1 0 1 0 
1110 0 110 
0 0 1 0 0 0 1 0 
0 10 110 0 0 
0 10 110 0 0 
1 0 0 0 0 0 1 0 
1 0 0 0 0 1 0 1 


= a 3 , 



and 

0000010100 
0000100001 
0 0 1 0 0 0 1 0 1 0 

1 0 1 0 0 0 0 0 1 0 

1 1 1 0 0 0 0 1 0 1 

None of the Rules 1-5 can be applied on the matrix 

A 3 =Remove 0 |A,, 8], Additionally, only Rule 4 can be 

applied to Remove 1 [A 1; 8 ] because 4 r/l column of this 

65 matrix, which is also the 4' , ' column of the original matrix 
Aq, is an all-zero vector. Applying this rule provides the 
following matrix: 
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'0 0 0 0 0 
0 0 0 0 1 
0 0 10 0 
10 10 0 
.11100 


10100’ 
0 0 0 0 1 

0 10 10 

0 0 0 1 0 

0 0 10 1 . 


5 


Therefore, the labels of the children of Node 1 are to 

X 3 =(A 3 , 0, {8, 9}), and 


X 4 =(A 4 , {9}, {4, 8}). 

(4.5.3) Node 2 15 

The label for Node 2 is X 2 . The updated lower and upper 

bounds are Lower_Bound2=4 and U=Upper_Bound=4. 
Since Lower_Bound=U and the function Test_Leaf [ k 2 , 
U]=True, this node is a leaf of the search tree. Thus, no 
further splitting is done with this node. 20 

(4.5.4) Node 3 

The label for Node 3 is X 3 . The updated lower and upper 
bounds are Lower_Bound=2 and U=Upper_Bound=3. The 
upper bound is realized with the set {6, 11, 12}, and the 
function Test_Leaf [X 3 , U]=False. Thus, this node has two 2; ' 
children (i.e., Node 5 and Node 6). Column 9 of the matrix 
A 3 , which is the \ Y h column of the original matrix A 0 , has 
the maximum weight. Then, 


Remove_0[4 3 , 9] : 


0 

0 

0 

0 

0 

0 

0 

1 

1 


u 


0 0 0 0 1 0 1 0 
0 0 0 1 0 0 0 1 
0 0 110 0 0 1 
0 1 0 0 0 1 0 0 
0 1110 0 10 
1 0 0 1 0 0 0 0 
10 10 110 0 
0 0 10 110 0 
0 1 0 0 0 0 0 0 
11000011 , 


30 


35 


40 


and 


Remove_l[/l 3 , 9] = 


0 

0 

0 

1 


U 


0 0 0 0 1 0 
0 0 0 1 0 0 
10 10 11 
0 0 10 11 
1 1 0 0 0 0 


1 0 
0 1 
0 0 
0 0 
1 lj 


45 


50 


None of the Rules 1-5 can be applied on the matrix 

Remove 0 [A 3 , 9] and Remove 1 [A 3 , 9]. Therefore, the 

labels of the children of Node 3 are 

55 

X 5 =(A 5 , 0, {8, 9. 11}). and 


X 6 =(A 6 . {11}. {8, 9}). 

(4.5.5) Node 4 

Hie label for Node 4 is X 4 . The updated lower and upper 
bounds are Lower_Bound=3 and U=Upper_Bound=3. Since 
Lower_Bound=U and Test_Leaf [X 4 , U]=True, this node is 
a leaf of the search tree. 

(4.5.6) Node 5 

Hie label for Node 5 is k 5 . The updated lower and upper 
bounds are Lower_Bomid=3 and U=Upper_Bound=3. Since 


20 

the Lower_Bound=U and the function Test_Leaf [X 5 , 
U]=True, this node is a leaf of the search tree. 

(4.5.7) Node 6 

The label for Node 6 is k 6 . The updated lower and upper 
bounds are Lower_Bound=3 and U=Upper_Bound=3. Since 
the Lower_Bound=U and the function Test_Leaf |X 6 , 
U]=True, this node is a leaf of the search tree. 

Therefore, the minimal Hitting Set in this example is {6, 
11, 12} which is produced by Node 3. FIG. 9 is an illustra- 
tion of an exemplary search tree 900, depicting the nodes 
and leafs of the above non-limiting example. 

(4.6) Mapping other Problems into the Integer Program- 
ming Problem 

The optimization problem formulated in system (1) not 
only covers the Hitting Set Problem, but it also covers 
several other important combinatorial optimization prob- 
lems. For example, it can also be applied to the Set Covering 
Problem and the Vertex Cover Problem. 

(4.6.1) The Set Covering Problem 

The Set Covering Problem is formulated as follows. A 
collection S={S 15 S 2 , . . . , S„} of subsets of a set X={x 1 , 
x 2 , . . . , x„,} is given. Using the Set Covering Problem, a user 
can find the minimum number of sets from the collection S 
such that their union is the background set X. In this case, 
the rows of the incidence matrix A are labeled by the 
elements x 1; x 2 , . . . , x„, and the columns by the sets S 1; 
S,, . . . , S„. Thus, one skilled in the art can appreciate that 
the solution of the optimization problem (1) is indeed the 
solution of the Set Covering Problem. 

(4.6.2) The Vertex Covering Problem 

The Vertex Covering Problem is formulated as follows. 
Given a graph G=(V, E), with the vertices V={v 1 , v 2 , . . . , 
v )( } and edges E={e 1 , e 2 , . . . , e m }> find a minimum size set 
of vertices U V such that for every edge e^EE, at least one 
of the endpoints belongs to the set U. FIG. 10 illustrates a 
graph 1000 and its vertex cover, with the set of black 
vertices 1002 being a minimal covering of this graph 1000. 

In this problem, the rows of the incidence matrix A are 
labeled by the edges e 1; e 2 , . . . , e m and the columns by the 
vertices v 1; v 2 , . . . , v„. In the row which is labeled by the 
edge e-, there are exactly two entries equal to 1 and those 
correspond with the endpoints of the edge e^.. As can be 
appreciated by one skilled in the art, the solution of the 
optimization problem (1) is again the solution of the Vertex 
Cover Problem. 

For further illustration, FIG. 11 is a graph 1100 showing 
the performance of the algorithms of the present invention 
(i.e., “new” 1102) contrasted with the traditional one (i.e., 
“old” 1104) for solving this problem. In this example, the 
algorithms are applied to 100 random graphs with 30 
vertices and different number of edges, with the average 
time of each algorithm being presented. As shown in FIG. 
11, the algorithm of the present invention is substantially 
faster in solving complex problems than traditional 
approaches. 

(4.7) Summary and Conclusions 

The present invention provides a new approach to over- 
come one of the major limitations of the current model- 
based diagnosis techniques (i.e., the exponential complexity 
of calculation of minimal diagnosis set). To overcome this 
challenging limitation, the present invention describes an 
algorithmic approach for calculating the minimal diagnosis 
set. Starting with the relationship between the calculation of 
minimal diagnosis set and the Hitting Set problem, the 
present invention includes a method for solving the Hitting 
Set Problem, and consequently the diagnosis problem. Hiis 
method is based on a powerful and yet simple representation 
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of the problem that enables its mapping onto another well- 
known problem, that is, the 0/1 Integer Programming prob- 
lem. 

Mapping onto the 0/1 Integer Programming problem 
enables the use of variety of algorithms that can efficiently 5 
solve the problem for up to several thousand components. 
Therefore, the algorithm of the present invention provides a 
significant improvement over existing ones, enabling the 
efficient diagnosis of large, complex systems. In addition, 
this mapping enables a priori and fast determination of the to 
lower and upper bounds on the solution (i.e., the minimum 
number of faulty components) before solving the problem. 
This is exploited to develop a more powerful algorithm for 
the problem. The new algorithm is an improvement of the 
well-known Branch-and-Bound method. The results of the 15 
performance of the new algorithm are presented on a set of 
test cases. The results show the advantage of the new 
algorithm over the traditional Branch-and-Bound algorithm. 
More specifically, the algorithm of the present invention has 
achieved more than 10 times the speed of standard algo- 20 
rithms. 

What is claimed is: 

1. A method for calculating a minimal Hitting Set from a 
collection of sets, comprising using a computer system to 
analyze a system having a collection of sets to derive the 25 
minimal Hitting Set from the collection of sets, the computer 
system being configured to perform acts of: 

receiving information regarding the collection of sets 
from the system; 

creating an incidence matrix of the collection of sets; 30 

creating nodes from the incidence matrix such that each 
node is a node in a search tree and has a label that 
includes a sub-matrix of the incidence matrix, and two 
disjoint subsets of the columns of the incidence matrix; 

calculating an upper bound and calculating a lower bound 35 
for each label, where the upper bound and lower bound 
define a range of solutions for the minimal Hitting Set; 
and 

determining whether each label has any child nodes or is 
a leaf node without any child nodes; 40 

if the label does not have any child nodes, then stop and 
designate the label as a leaf node; 
if the label does have a child node, splitting the label to 
create two new labels; 

repeating acts of calculating and determining until all 45 
labels have been designated as leaf nodes; 
determining a solution of the Hitting Set with informa- 
tion gathered from leaf nodes; and 
providing an indication of the Hitting set to a user. 

2. A method for calculating a minimal Hitting Set as set 50 
forth in claim 1, wherein in the act of calculating the 
minimal Hitting Set, the minimal Hitting Set is a minimal 
diagnosis set and the collection of sets is a collection of 
conflict sets, such that the method for calculating a minimal 
Hitting Set calculates a solution for a diagnosis problem. 55 

3. A method for calculating a minimal Hitting Set as set 
forth in claim 2, wherein in the act of creating an incidence 
matrix, the incidence matrix is created as a binary matrix 
denoted as mxn binary matrix A having columns and rows, 
where the columns of matrix A are labeled by numbers 1 , 60 
2, . . . ,n, and where any subset of these columns are denoted 

as a subset of {1,2, . . . ,n}. 

4. A method for calculating a minimal Hitting Set as set 
forth in claim 3, wherein in the act of creating nodes, each 
node has a label (X) in a form of X=(M, T,-„, T out ), where M 65 
is a sub-matrix of matrix A, and where T,„ and T out are 
disjoint subsets of the columns of matrix A, where T,„ 
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denotes a set of columns of matrix A considered as part of 
the minimal diagnosis set, and where T out denotes a set of 
columns of matrix A that are considered not as part of the 
minimal diagnosis set. 

5. A method for calculating a minimal Hitting Set as set 
forth in claim 4, wherein the act of calculating an upper 
bound further comprises acts of calculating a value for the 
upper bound of X=(M, T,„, T our ) and a calculating an upper 
bound set, by performing the following acts: 

initializing the value of the upper bound as U=size of T,„ 
and initializing the upper bound set as the set T,„, where 
the size of T,„ is the number of elements in T !M ; 

choosing a column a 1 of M with a maximum weight, 
where the column with the maximum weight is the 
column of M with the largest number of l’s; 

increasing the value of U by 1 and adding the column a x 
to the upper bound set; 

constructing the matrix M x from M by deleting the col- 
umn a 3 and all rows of M that correspond with non- zero 
components of the column a l5 where if the matrix M x 
is empty then the current value of U is the upper bound 
and a solution for the Hitting Set is the upper bound set; 
otherwise 

repeating the acts of choosing through repeating on the 
matrix Mj . 

6. A method for calculating a minimal Hitting Set as set 
forth in claim 5, wherein in the act of calculating a lower 
bound, a value for the lower bound of X=(M, T,„, T out ) is 
equal to the size of T,„ plus k/L, where k is the number of 
rows of the matrix M and L is the maximum weight of the 
columns of M. 

7. A method for calculating a minimal Hitting Set as set 
forth in claim 6, wherein in the act of determining whether 
each label has any child nodes or is a leaf node without any 
child nodes, a label is determined to be a leaf node if any of 
the following are true: 

if T ( „UT ou ,={l,2, . . . ,n}, where U denotes the union of 
two sets; 

if the value for the lower bound of the current value 
for the upper bound; 

if the columns of T,„ form a solution such that if the 
columns in T irl are added together to form a resulting 
vector, then the resulting vector has no zero compo- 
nent; or 

if M contains an all-zero row. 

8. A method for calculating a minimal Hitting Set as set 
forth in claim 7, wherein the act of splitting the label to 
create two new labels further comprises an act of splitting 
label X=(M, T 1M , T out ) into label X 0 and label X t ; 

where label X 0 is defined as: 

X) = (Remove 13 0 [M, jj, T,.„, T„„,U{j}), 

where j denotes a column of M with a maximum weight and 
J is the corresponding column in the original matrix, and 
where Remove„0 [M, j] denotes a sub-matrix of M obtained 
by deleting the \‘ h column of M; and 

where label X 3 is defined as: 

Xj=(Remove_l [M, j], T,.„U{j}, T„J, 

where j denotes a column of M with a maximum weight and 
j is the corresponding column in the original matrix, and 

Remove 1 [M, j] denotes a sub-matrix of M obtained by 

deleting the j'* column of M and deleting all rows of M that 
correspond with non-zero components of that column. 

9. A method for calculating a minimal Hitting Set as set 
forth in claim 8, wherein the act of splitting a label further 
comprises acts of: 
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on label X=(M, T,„, T out ), if the j* column of the matrix 
M is an all-one column, then changing the label X to the 
label ({ }, T,-„U{j }, T out ), where U denotes the union of 
two sets, { } is an empty matrix and j is a corresponding 
column in the original matrix; 5 

on label X=(M, T,„, T out ), if the matrix M has a row of 
weight one, with 1 as its j* component, then changing 
the label X to the label (Remove__l [M, j], T,„U{j}, 

T ' out ) where U denotes the union of two sets, j is the 
corresponding column in the original matrix, and to 

Remove 1 [M, j] denotes a sub-matrix of M obtained 

by deleting the j* column of M and deleting all rows of 
M that correspond with non-zero components of that 
column; 

on label X=(M, T !K , T out ), if the matrix M has an all-one 15 
row, then deleting that row; 

on label X=(M, T in , T out ), if j* column of the matrix M is 
an all-zero column, then changing the label X to the 
label (Remove — 0 [M, j], T,.„, T„JJ{j}), where U 
denotes the union of two sets, j is the corresponding 20 

column in the original matrix, and Remove 0 [M, j] 

denotes a sub-matrix of M obtained by deleting the j' ; ' 
column of M; 

on label X=(M, T iM , T out ), if the matrix M has two equal 
rows, then deleting one of them; and 25 

repeating the acts of claim 9 until all of the acts of claim 
9 are exhausted. 

10. A method for calculating a minimal Hitting Set as set 
forth in claim 9, further comprising an act of applying a 
Branch-and-Bound technique to the incidence matrix A to 30 
find an optimal solution for the problem, comprising acts of: 

initializing a label X 0 as X 0 =Rules*[{A, { }, { }}], where 
Rules* denotes applying and repeating the acts of claim 
9 until all of the acts of claim 9 are exhausted; 
initializing a set of labels as {X 0 }; 35 

initializing a value of the upper bound U as the upper 
bound of X 0 ; 

initializing a solution set as the upper bound set of X 0 ; 
choosing a label X from the set of labels; 
deleting X from the set of labels; 40 

computing Uj as the upper bound of X and its correspond- 
ing upper bound set; 

if Uj<U, then changing the value of U to U\ and the value 
of the solution set to the upper bound set; 
if the label X is not a leaf, then splitting X to labels X 0 and 45 
Xj and adding the new labels X 0 and Xj to the set of 
labels; 

if the set of labels is not empty, repeating the acts of 
choosing a label X through the act of repeating the acts; 
if the set of labels is empty, then the outputting is the 50 
solution set and stopping. 

11 . A computer program product for calculating a minimal 
Hitting Set from a collection of sets, the computer program 
product comprising computer-readable instruction means 
stored on a computer-readable medium for causing a com- 55 
puter to: 

receive information regarding the collection of sets from 
the system; 

create an incidence matrix of the collection of sets; 
create nodes from the incidence matrix such that each 60 
node is a node in a search tree and has a label that 
includes a sub-matrix of the incidence matrix, and two 
disjoint subsets of the columns of the incidence matrix; 
calculate an upper bound and calculating a lower bound 
for each label, where the upper bound and lower bound 65 
define a range of solutions for the minimal Hitting Set; 
and 
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determine whether each label has any child nodes or is a 
leaf node without any child nodes; 
if the label does not have any child nodes, then stop and 
designate the label as a leaf node; 
if the label does have a child node, splitting the label to 
create two new labels; 

repeating operations of calculating and determining 
until all labels have been designated as leaf nodes; 
determining a solution of the Hitting Set with informa- 
tion gathered from leaf nodes; and 
providing an indication of the Hitting set to a user. 

12. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 11, wherein when 
calculating the minimal Hitting Set, the minimal Hitting Set 
is a minimal diagnosis set and the collection of sets is a 
collection of conflict sets, such that the computer program 
product is configured to calculate a solution for a diagnosis 
problem. 

13. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 12, wherein when 
creating an incidence matrix, the incidence matrix is created 
as a binary matrix denoted as mxn binary matrix A having 
columns and rows, where the columns of matrix A are 
labeled by numbers 1,2,... ,n, and where any subset of these 
columns are denoted as a subset of {1,2, . . . ,n}, and 

wherein when creating nodes, each node has a label (X) in 
a form of X=(M, T tn , T out ), where M is a sub-matrix of 
matrix A, and where T,„ and T out are disjoint subsets of 
the columns of matrix A, where T in denotes a set of 
columns of matrix A considered as part of the minimal 
diagnosis set, and where T ou , denotes a set of columns 
of matrix A that are considered not as part of the 
minimal diagnosis set. 

14. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 13, wherein when 
calculating an upper bound, the computer program product 
is further configured to cause a computer to perform opera- 
tions of calculating a value for the upper bound of X=(M, T in , 
T out ) and a calculating an upper bound set, by performing 
the following operations: 

initializing the value of the upper bound as U=size of lj„ 
and initializing the upper bound set as the set T iM , where 
the size of T,„ is the number of elements in T,„; 

choosing a column aj of M with a maximum weight, 
where the column with the maximum weight is the 
column of M with the largest number of l’s; 

increasing the value of U by 1 and adding the column a : 
to the upper bound set; 

constructing the matrix M t from M by deleting the col- 
umn a t and all rows of M that correspond with non- zero 
components of the column a 1; where if the matrix M : 
is empty then the current value of U is the upper bound 
and a solution for the Hitting Set is the upper bound set; 
otherwise 

repeating the operations of choosing through repeating on 
the matrix M,. 

15. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 14, wherein when 
calculating a lower bound, a value for the lower bound of 
X=(M, Tj„, T out ) is equal to the size of T,„ plus k/L, where 
k is the number of rows of the matrix M and L is the 
maximum weight of the columns of M. 

16. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 15, wherein when 
determining whether each label has any child nodes or is a 
leaf node without any child nodes, the computer program 
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product is configured to cause a computer to determine a 
label to be a leaf node if any of the following are true: 

if T,„UT ou ,={l,2, . . . ,n}, where U denotes the union of 
two sets; 

if the value for the lower bound of X = the current value 
for the upper bound; 

if the columns of T in form a solution such that if the 
columns in T,„ are added together to form a resulting 
vector, then the resulting vector has no zero compo- 
nent; or 

if M contains an all -zero row. 

17. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 16, wherein when 
splitting the label to create two new labels, the computer 
program product is further configured to cause a computer to 
perform an operation of splitting label X=(M, T ln , T ou J into 
label X 0 and label k 1 ; 

where label X 0 is defined as: 

X 0 =(Remove_0 [M, j], T,„, T„„,U{j}), 

where j denotes a column of M with a maximum weight and 
j is the corresponding column in the original matrix, and 
where Remove_0 [M, j] denotes a sub-matrix of M obtained 
by deleting the j th column of M; and 

where label Xj is defined as: 

X 1 =(Remove_l [M, j], T,.„U{j}, T M ), 

where j denotes a column of M with a maximum weight and 
j is the corresponding column in the original matrix, and 
Remove_l [M, j] denotes a sub-matrix of M obtained by 
deleting the column of M and deleting all rows of M that 
correspond with non-zero components of that column. 

18. A computer program product for calculating a mini- 
mal Hitting Set as set forth in claim 17, wherein when 
splitting a label, the computer program product further 
comprises instruction means for causing a computer to 
perform operations of: 

on label X=(M, T in , T ou t ), if the ]‘ h column of the matrix 
M is an all-one column, then changing the label X to the 
label ({ }, T 1 m U{j }, T out ), where U denotes the union of 
two sets, { } is an empty matrix and j is a corresponding 
column in the original matrix; 

on label X=(M, Tj.„, T ou ,), if the matrix M has a row of 
weight one, with 1 as its j r,i component, then changing 

the label X to the label (Remove 1 [M, j], T ft; U{j}, 

Tout) where U denotes the union of two sets, j is the 
corresponding column in the original matrix, and 

Remove 1 [M, j] denotes a sub-matrix of M obtained 

by deleting the ] ,h column of M and deleting all rows of 
M that correspond with non-zero components of that 
column; 

on label X=(M, T f „, T our ), if the matrix M has an all-one 
row, then deleting that row; 

on label X=(M, T ; „, T ou ,), if j' 7 ' column of the matrix M is 
an all-zero column, then changing the label X to the 
label (Remove — 0 [M, j], T ; „, T„„,U{j}), where U 
denotes the union of two sets, j is the corresponding 

column in the original matrix, and Remove 0 [M, j] 

denotes a sub-matrix of M obtained by deleting 
column of M; 


26 

on label X=(M, T„„ T our ), if the matrix M has two equal 
rows, then deleting one of them; and 
repeating operations of claim 19 until all of the operations 
of claim 19 are exhausted. 

5 19. A computer program product for calculating a mini- 

mal Hitting Set as set forth in claim 18, further comprising 
instruction means for causing a computer to perform an 
operation of applying a Branch-and-Bound technique to the 
incidence matrix A to find an optimal solution for the 
1° problem, comprising operations of: 

initializing a label X 0 as X 0 =Rules*[{A, { }, { }}], where 
Rules* denotes applying and repeating the operations 
of claim 19 until all of the operations of claim 19 are 
exhausted; 

15 initializing a set of labels as {X 0 }; 

initializing a value of the upper bound U as the upper 
bound of X 0 ; 

initializing a solution set as the upper bound set of X 0 ; 
choosing a label X from the set of labels; 
deleting X from the set of labels; 

computing Uj as the upper bound of X and its correspond- 
ing upper bound set; 

if Uj<U, then changing the value of U to Uj and the value 
25 of the solution set to the upper bound set; 

if the label X is not a leaf, then splitting X to labels X 0 and 
Xj and adding the new labels X 0 and X x to the set of 
labels; 

if the set of labels is not empty, repeating the operations 
30 of choosing a label X through the operation of repeating 

the operations; 

if the set of labels is empty, then the outputting is the 
solution set and stopping. 

20. A computer program system for calculating a minimal 
35 diagnosis set from a collection of conflict sets, the computer 
system being configured to perform operations of: 
creating an incidence matrix of the collection of sets; 
creating nodes from the incidence matrix such that each 
40 node is a node in a search tree and has a label that 
includes a sub-matrix of the incidence matrix, and two 
disjoint subsets of the columns of the incidence matrix; 
calculating an upper bound and calculating a lower bound 
for each label, where the upper bound and lower bound 
45 define a range of solutions for the minimal Hitting Set; 
and 

determining whether each label has any child nodes or is 
a leaf node without any child nodes; 
if the label does not have any child nodes, then stop and 
50 designate the label as a leaf node; 

if the label does have a child node, splitting the label to 
create two new labels; 

repeating acts of calculating and determining until all 
labels have been designated as leaf nodes; 

55 determining a solution of the Hitting Set with information 
gathered from leaf nodes; and 
providing an indication of the Hitting set to a user. 



